Comparing Support Vector Machines (SVMs) and Bayesian Networks (BNs) in detecting driver cognitive distraction using eye movements

Driver distraction is an important and growing safety concern as information technologies, such as navigation systems and internet-content services, have become increasingly common in vehicles. To allow people to benefit from these technologies without compromising safety, an adaptive In-Vehicle Information System (IVIS) is needed. Such systems can manage drivers workload and mitigate distraction by monitoring and responding to driver states and roadway conditions. For the system to be effective, however, it is critical that driver distraction can be identified accurately in real time. This chapter discusses approaches to identify driver cognitive distraction. Eye movements and driving performance were chosen as promising indicators. A robust data fusion system using data mining techniques was proposed to integrate these indicators to detect when a driver was distracted. The objective of this chapter is to compare two data mining methods for identifying driver cognitive distraction using eye movements and driving performance, Support Vector Machines (SVMs) and Bayesian Networks (BNs), respectively. We trained and tested Dynamic BN (DBN), Static BN (SBN), and SVM models with experimental data and assessed model performance using testing accuracy and two single-detection-theory measures. The DBN method, which considers behavior change over time, produced the most accurate and sensitive models. The DBNs and SVM models were more accurate and had higher hit rate than the SBN models. These results indicate that both sequential changes in eye movements and driving performance are important predictors of drivers cognitive distraction. If time-dependent relationships are ignored the SVM method has advantages over the SBN method because of its computational ease and flexible parameter-choosing strategy. Thus, a hybrid method combining the DBN and SVM methods may create models that perform better than any of three types of models that have been developed to date. Driver distraction is an important safety problem [28]. The results of a study that tracked 100 vehicles for one year indicated that nearly 80% of crashes and 65% of near-crashes involved some form of driver inattention within three seconds of the event. The most common form of inattention included secondary tasks, drivingrelated inattention, fatigue, and combinations of these [14]. In-vehicle information systems (IVIS), such as navigation systems and internet services, introduce various secondary tasks into the driving environment that can increase crash risk [1, 27]. Thus, in future it would be very beneficial if IVIS could monitor driver distraction so that the system could adapt and mitigate the distraction. To not disturb driving, non-intrusive and real-time monitoring of distraction is essential. This chapter begins to address this issue by describing techniques that draw upon the data from a videobased eye tracking system to estimate the level of drivers cognitive distraction in real time. First, three types of distractions and the detection techniques are briefly described. Then, the chapter focuses on the cognitive distraction and brings out a general procedure for implementing a detection system for such distraction. Data mining methods are proposed to be promising techniques to infer a drivers cognitive state from their behavior. Next, Support Vector Machines (SVMs) and Bayesian Networks (BNs) are used and compared in this application. Finally, several issues associated with the detection of cognitive distractions are discussed. 1 Types of distraction Three major types of distraction have been widely studied: visual, manual, and cognitive [28]. These distractions deflect drivers visual attention, manual operation, and cognitive resources away from the driving control task, respectively, and result in the degradation of driving performance and even cause fatal accidents. Visual distraction and manual distraction can be directly observed through the external behaviors of drivers, such as glancing at billboards or releasing the steering wheel to adjust the radio. Visual distraction usually coexists with manual distraction because visual cues provide necessary feedback when people perform manual tasks. Visual and manual distractions interrupt continuous visual perception and manual operation essential for driving and results in the absence of visual attention on safety-critical events. In the 100-vehicle study, visual inattention contributed to 93% of rear-end-striking crashes. Interestingly, in eighty-six percent of the rear-end-striking crashes, the headway at the onset of the event the led to the crash was greater than 2.0 seconds [14]. These facts show that visual distraction dramatically increases real-end-striking crashes because two seconds are long enough for an attentive driver to avoid a collision. The degree of visual distraction is proportional to the eccentricity of visual-manual tasks to the normal line of sight [16]. Unlike visual and manual distraction, cognitive distraction is internal and impossible to observe from external behavior. With visual distraction it is possible to detect when the eyes are off the road, but with cognitive distraction there is no direct indicator as to when the mind is off the road. Nonetheless, the effects of cognitive distraction on driving performance are as negative as those of visual distraction. A meta-analysis of twenty-three studies found that cognitive distraction delayed driver response to hazards [10]. For example, drivers reacted more slowly to brake events [16, 17] and missed more traffic signals [32] when they performed mental tasks while driving, such as using auditory e-mail systems, performing math calculation, or holding hand-free cell-phone-conversations. To identify how visual distraction delays reaction time several researchers have created predictive models that quantify the risks associated with visual or manual distraction from drivers glance behavior. It was found that frequent glances to a peripheral display caused drivers to respond slowly to breaking vehicles ahead of them. The reaction time, the time from the time the lead vehicle began to brake until the driver released the accelerator, could be predicted by the proportion of off-road glances in this reaction period using a linear equation: (accelerator − releasereactiontime) = 1.654 + 1.581(off − roadproportion) [38]. This relationship accounts for 50% of reaction-time variance. Another study took historic performance into account. It used a linear function of current glance duration away from road, β1, and total glance duration away from road during the last three seconds, β2, to calculate warning threshold, γ, as: γ = αβ1 + (1− α)β2. γ influenced the frequency of alarms for reminding drivers when they were too engaged in a visual-manual task, and α presented the weights of the two glance durations on γ [7]. Using this equation, those drivers who had been identified as risky drivers received more warnings per minute than non-risky drivers for a broad range of α and γ. These studies found diagnostic measures, such as frequency and duration of off-road glances, and used these measures to predict the degree to which visual and manual distractions delayed drivers reaction time to critical roadway events. At the same time, these predictive models can be used to monitor drivers visual and manual distraction non-intrusively in real time because remote eye tracker cameras can collect and output eye movement data without disturbing driving. However, identifying cognitive distraction is much more complex and less straightforward than visual and manual distraction. There is no clearly diagnostic predictor for cognitive distraction. In controlled situations, four categories of measures are used to assess workload and cognitive distraction: subjective measures, secondary task performance, primary task performance, and physiological measures [37, 39]. Subjective measures and secondary task performance can not be used to identify cognitive distraction for future IVIS. Commonly-used subjective measures include the NASA Task Load Index (NASA-TLX) and the subjective workload assessment technique (SWAT). Collecting subjective measures disturbs normal driving and cannot provide an unobtrusive real-time indicator. The secondary task method for assessing workload would require the driver to perform a task in addition to driving and whatever interaction he or she might have with a in-vehicle system and so is clearly inappropriate for measuring distraction as an unobtrusive real-time indicator of distraction. Other measures, such as primary task performance and physiological measures, present promising predictors for real-time estimation of cognitive distraction. The primary task refers to driving control task. The commonly-used driving performance measures include lane position variability, steering error, speed variability, and so on. These measures can be collected non-intrusively using driving simulators or sensors on instrumented vehicles in real time. Physiological measures, such as heart-rate variability, pupil diameter, eye movements, represent promising sources of information regarding the drivers state. Although monitoring heart rate and pupil diameter in vehicles is difficult due to sensor limits, it is feasible to track eye movements of drivers in real time with advanced eye tracking systems. Eye movement patterns change with different levels of cognitive distraction. Cognitive distraction disrupts the allocation of visual attention across the driving scene and impairs information processing. Recarte and Nunes [25] found that increased cognitive load was associated with longer fixations, gaze concentration in the center of the driving scene, and less frequent glances at mirrors and at the speedometer. Cognitive distraction impaired the ability of drivers to detect targets across the entire visual scene [26, 35], and reduced implicit perceptual memory and explicit recognition memory for items that drivers fixated [31]. One study that systematically examined the sensitivity of various eye movement measures to the complexity of in-vehicle cognitive tasks found that standard deviation of gaze was the most sensitive indicator for the level of complexity [35]. Of the four categories of potential measures of cognitive distraction, eye movements and driving performance are the most suitable measures for estimating cognitive distraction [20, 21, 39]. Cognitive distraction presents a great risk in driving because it delays drivers response to critical events and is more difficult to detect than visual distraction. However, recent developments suggest eye movements and driving performance offer a promising basis for detecting cognitive distraction. Although cognitive, visual, and manual distractions are described separately they coexist in most situations. For example, when entering an address into a GPS while driving, drivers need to recall the address and then glance at the system to enter it. This leads to both a visual and a cognitive distraction. Ultimately algorithms that detect visual distraction and cognitive distraction will need to work together to provide comprehensive prediction of driver distraction. However, the balance of this chapter focuses on the challenging task of estimating cognitive distraction using eye movements and driving performance measures. 2 The process of identifying cognitive distraction Detecting cognitive distraction is complex procedure and requires a robust data fusion system. Unlike visual and manual distraction, the challenge of detecting cognitive distraction is to integrate multiple data streams, including eye movements and driving performance, in a logical manner to infer the drivers cognitive state. One way to address this challenge is by using data fusion. Data fusion systems can align data sets, correlate relative variables, and combine the data to make detection or classification decision [36]. One benefit of using a data fusion perspective to detect cognitive distraction is that data fusion can occur at different abstract levels. For instance, sensor data are aggregated to measure driver’s performance at the most concrete level, and then these performance measures can be used to characterize driver’s behavior at higher levels of abstraction. This hierarchical structure can logically organize the data and inferences and reduce parameter space in detection procedure. The fusion systems also can continuously refine the estimates made at each level across time, which enables a real-time estimation of cognitive distraction. To implement a data fusion system, there are two general approaches: top-down and bottom-up. The top-down approach identifies the targets based on the known characteristics, such as shape and kinematic behavior. In the detection of cognitive distraction, the top-down approach is presented by using drivers’ behavioral response of people under high levels of cognitive load that reflect existing theories of human cognition, such as Multiply Resource Theory [37] and ACT-R [29]. The limitation of the top-down approach is that it is impossible to implement data fusion without complete understandings of the underlying processłsomething that is lacking in the area of driver distraction. The bottom-up approach overcomes this limitation and uses data mining methods to extract characteristics of the targets from data directly. Data mining includes a broad range of approaches that can search large volumes of data for unknown patterns, using techniques such as decision trees, evolutionary algorithms, support vector machines, and Bayesian networks. These methods are associated with multiple disciplines (e.g., statistics, information retrieval, machine learning, and pattern recognition) and have been successfully applied in business and health care domains [3, 33]. In driving domain, decision tree, Support Vector Machines (SVMs), and Dynamic Bayesian Networks (DBNs) have successfully captured the differences in behavior between when people drive normally and when distracted and produced promising results in detecting cognitive distraction [20, 21, 39]. The strategies of constructing data fusion systems include using the top-down approach alone, the bottom-up approach alone, or the mixed approach that combines top-down and bottom-up. The choice of the strategies depends on the availability of domain knowledge, as shown in Table 1. When the targets are understood very well, the data fusion system can be constructed using only the top-down approach. Currently, most data fusion systems use this strategy. Nevertheless, the lack of domain knowledge presents an important constraint of this top-down-alone strategy in some domains, such as the detection of cognitive distraction. The bottom-up-alone and mixed strategies overcome the limitation. Oliver and Horvitz [24] Data fusion strategies Top-down approach Mixed approach Bottom-up approach Domain knowledge All available √ √ √ Partially available √ √